Designing a unified solution to the combinational optimization problems of undesigned heuristic algorithms has become a research hotspot in the field of machine learning. At present， mature technologies are mainly aiming at static combinatorial optimization problems， but the combinational optimization problems with dynamic changes are not fully solved. In order to solve above problems， a lightweight model called Dy4TSP （Dynamic model for Traveling Salesman Problems） was proposed， which combined multi-head-attention mechanism with distributed reinforcement learning to solve the traveling salesman problem on a dynamic graph. Firstly， the node representation vector from graph convolution neural network was processed by the prediction network based on multi-head-attention mechanism. Then， the distributed reinforcement learning algorithm was used to quickly predict the possibility that each node in the graph was output as the optimal solution， and the optimal solution space of the problems in different possibilities were comprehensively explored. Finally， the action decision sequence which could meet the specific reward function in real time was generated by the trained model. The model was evaluated on three typical combinatorial optimization problems， and the experimental results showed that the solution qualities of the proposed model are 0.15 to 0.37 units higher than those of the open source solver LKH3 （Lin-Kernighan-Helsgaun 3）， and are significantly better than those of the latest algorithms such as Graph Attention Network with Edge Embedding （EGATE）. The proposed model can reach an optimal path gap of 0.1 to 1.05 in other dynamic traveling salesman problems， and the results are slightly better.